Use reshape/ravel to set numpy shape #1165

sergey-yaroslavtsev · 2025-12-22T20:31:43Z

Closes #1162
Following suggestions from comments in the issue to switch all .shap= to either reshape or ravel.

New pattern using ndarray.reshape
arr = numpy.zeros((1,3,4));arr = arr.reshape(-1, 6);print(arr.shape)

New pattern using numpy.ravel
arr = numpy.array(0);arr=numpy.ravel(arr);print(arr.shape)

Fixed places are less than mentioned "485". This ".shape=" remained in .c files and in some comments, where for my understanding they should not be changed. Or am I wrong?
In .pyx files it was changed.

 pymca --test

runs OK.

Please have a brief look.

Note:
in many cases I used "*" to unpack the input shape but it is not required - now I doubt if it makes things clearer or worse.

b = numpy.array([1, 2, 3, 4, 5, 6, 7, 8])
b = b.reshape((2,4))
b
array([[1, 2, 3, 4],
       [5, 6, 7, 8]])
b = numpy.array([1, 2, 3, 4, 5, 6, 7, 8])
b = b.reshape([2, 4])
b
array([[1, 2, 3, 4],
       [5, 6, 7, 8]])
b = numpy.array([1, 2, 3, 4, 5, 6, 7, 8])
b = b.reshape(*(2,4))
b
array([[1, 2, 3, 4],
       [5, 6, 7, 8]])

woutdenolf

LGTM.

For reference:

Unlike numpy.reshape the instance method ndarray.reshape can take either a tuple but also an unpacked tuple. This PR mixes both conventions. I don't mind.
reshape and ravel only copies the array when needed. In contrast flatten (which we do not use in this PR) always copies.

As for the cython generated code, I think we should do it there as well (not manually). But that can be another PR.

For example this will break in numpy 2.4.0 because to the strides trick:

} else {__pyx_pybuffernd_mask.diminfo[0].strides = __pyx_pybuffernd_mask.rcbuffer->pybuffer.strides[0]; __pyx_pybuffernd_mask.diminfo[0].shape = __pyx_pybuffernd_mask.rcbuffer->pybuffer.shape[0];

See for example silx-kit/pyFAI#2711.

These .c files say /* Generated by Cython 0.29.32 */. Not sure what's going on here. Why are they in the repo @vasole ?

Edit: it seems like setup.py anticipates cython not being available in which case it falls back to the c files. We installed cython and re-generated the .c files but no changes were applied, even when numpy 2.4.0 was installed.

woutdenolf · 2025-12-23T08:56:17Z

src/PyMca5/PyMcaCore/StackBase.py

                              numpy.sum(tmpData, 2),
                              self._stackImageData[i:i+step,:])
-                    tmpData.shape = step*shape[1], shape[2]
+                    tmpData = tmpData.reshape(step*shape[1], shape[2])


Ok I did not know that elements of the shape parameter can be passed in as separate arguments.

https://numpy.org/devdocs/reference/generated/numpy.ndarray.reshape.html

FYI The free numpy.reshape function only accepts a tuple.

but you did it in the suggestion for issue #1162

New pattern using ndarray.reshape
arr = numpy.zeros((1,3,4));arr = arr.reshape(-1, 6);print(arr.shape)
(2, 6)

and to be concistent between
arr.reshape(X, Y) + arr.reshape(*tuple)
or
arr.reshape((X, Y)) + arr.reshape(tuple)
I chose 1st option

Indeed. I just didn't know before. I've only used the free numpy.reshape until I looked at the easiest way to refactor setting shape.

woutdenolf · 2025-12-23T09:01:17Z

src/PyMca5/PyMcaGui/physics/xrf/McaAdvancedFit.py

            else:
                ddict['result'][label]  = ymatrix
-            ddict['result'][label].shape  = (len(ddict['result'][label]),)
+            ddict['result'][label] = numpy.ravel(ddict['result'][label])


That's indeed another case where we can use numpy.ravel.

@sergey-yaroslavtsev We introduced silent flattening here. See #1170 (comment).

src/PyMca5/PyMcaGui/math/NNMADialog.py

woutdenolf · 2025-12-23T09:07:26Z

src/PyMca5/PyMcaGui/pymca/SilxScatterWindow.py

                if not gotIt:
                    raise ValueError("Unmatched dimensions following C order")
-                data.shape = xsize, oldDataShape[i+1:]
+                data = data.reshape(xsize, *oldDataShape[i+1:])


Good catch. This was a bug.

woutdenolf · 2025-12-23T09:10:58Z

src/PyMca5/PyMcaIO/SupaVisioMap.py

        ncols = values[2]
        self.nSpectra = nrows * ncols
-        data.shape = [len(data)/3, 3]
+        data = data.reshape(int(len(data)/3), 3)


woutdenolf · 2025-12-23T09:14:48Z

src/PyMca5/PyMcaPhysics/xrf/LegacyFastXRFLinearFit.py

                    else:
                        spectra = data[badMask, iXMin:iXMax+1]
-                    spectra.shape = badMask.sum(), -1
+                    spectra = spectra.reshape(int(badMask.sum()), -1)


Good catch.

Co-authored-by: Wout De Nolf <[email protected]>

vasole · 2025-12-23T18:26:41Z

Thank you!

woutdenolf

I propose the fix the three silent flattenings we introduced here. See #1170 (comment) for the preferred way.

woutdenolf · 2026-01-07T21:44:59Z

src/PyMca5/PyMcaGui/physics/xrf/McaAdvancedFit.py

            else:
                ddict['result'][label]  = ymatrix
-            ddict['result'][label].shape  = (len(ddict['result'][label]),)
+            ddict['result'][label] = numpy.ravel(ddict['result'][label])


@sergey-yaroslavtsev We introduced silent flattening here. See #1170 (comment).

woutdenolf · 2026-01-07T21:46:04Z

src/PyMca5/PyMcaMath/mva/Lanczos.py

    def diago(self, k, m):
        mat = numpy.zeros([m,m], numpy.float64)
-        mat.shape=[m*m]
+        mat = numpy.ravel(mat)


@sergey-yaroslavtsev We introduced silent flattening here. See #1170 (comment).

woutdenolf · 2026-01-07T21:52:32Z

src/PyMca5/PyMcaGui/physics/xrf/McaAdvancedFit.py

-                xdata.shape= [len(xdata),]
-                ydata.shape= [len(ydata),]
+                xdata = numpy.ravel(xdata)
+                ydata = numpy.ravel(ydata)


@sergey-yaroslavtsev We introduced silent flattening here. See #1170 (comment).

sergey-yaroslavtsev added 4 commits December 22, 2025 19:08

reshape/ravel

5f33a60

reshape/ravel part2

1547b3b

reshape/ravel part3

c064e6b

unfold tuples where forgoten

74f95b3

sergey-yaroslavtsev requested review from vasole and woutdenolf December 22, 2025 20:31

woutdenolf approved these changes Dec 23, 2025

View reviewed changes

Update src/PyMca5/PyMcaGui/math/NNMADialog.py

65b091f

Co-authored-by: Wout De Nolf <[email protected]>

vasole merged commit db58840 into master Dec 23, 2025
6 checks passed

vasole deleted the numpy_shape branch December 23, 2025 18:26

sergey-yaroslavtsev mentioned this pull request Jan 7, 2026

Fix remain arr.shape= #1170

Open

woutdenolf reviewed Jan 7, 2026

View reviewed changes

Use reshape/ravel to set numpy shape #1165

Use reshape/ravel to set numpy shape #1165

Uh oh!

Conversation

sergey-yaroslavtsev commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

woutdenolf left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

woutdenolf Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

woutdenolf Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vasole commented Dec 23, 2025

Uh oh!

woutdenolf left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sergey-yaroslavtsev commented Dec 22, 2025 •

edited

Loading

woutdenolf left a comment •

edited

Loading

woutdenolf Dec 23, 2025 •

edited

Loading

woutdenolf Dec 23, 2025 •

edited

Loading